feat(metrics): emit hoodie.table.version as gauge on client init#19108
Draft
shangxinli wants to merge 3 commits into
Draft
feat(metrics): emit hoodie.table.version as gauge on client init#19108shangxinli wants to merge 3 commits into
shangxinli wants to merge 3 commits into
Conversation
Adds HoodieMetrics#emitTableVersionMetric(int) which registers a "table.tableVersion" gauge in the metrics registry. BaseHoodieClient calls it once per client construction after loading the table's metaClient. Motivation ---------- The existing metrics surface does not expose the on-disk Hudi table version at all. Operators upgrading across major Hudi versions (particularly anyone running with hoodie.write.auto.upgrade=false and a pinned hoodie.write.table.version) currently have no observability signal to alert when a table is unexpectedly promoted to a newer table version by an upgrade path. This change fills the gap with a single one-shot gauge that fires on every write / table-service client init, so standard alerting (Prometheus / Datadog / Graphite / JMX) can flag the drift in near real time. Behavior -------- - Gauge name: <metric.prefix>.<tableName>.table.tableVersion - Value: integer HoodieTableVersion#versionCode() (e.g. 6, 8) - Emitted at the end of BaseHoodieClient construction, once. - Best-effort: if the table has not yet been initialized (first-ever write to a new base path), the metaClient load throws and we log at DEBUG and skip; the gauge will surface on the next client init after the table exists. - No-op when metrics are disabled (config.isMetricsOn() == false). Test plan --------- Added two unit tests to TestHoodieMetrics: - testEmitTableVersionMetric: asserts the gauge is registered with the supplied version code. - testEmitTableVersionMetricWhenMetricsDisabled: asserts no-op when metrics are off. Compatibility ------------- Pure additive change. No on-disk state, no schema change, no behavior change for tables that disable metrics. Existing callers unaffected.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe the issue this Pull Request addresses
The metrics surface does not expose the on-disk Hudi table version. Operators upgrading across major Hudi versions — particularly anyone running with
hoodie.write.auto.upgrade=falseand a pinnedhoodie.write.table.version— currently have no observability signal to alert when a table is unexpectedly promoted to a newer table version. This change fills that gap with a one-shot gauge emitted on every write / table-service client init, so standard alerting (Prometheus / Datadog / Graphite / JMX) can flag the drift in near real time.Summary and Changelog
HoodieMetrics#emitTableVersionMetric(int)which registers atable.tableVersiongauge in the metrics registry.BaseHoodieClientcalls it once per client construction after loading the table's metaClient.config.isMetricsOn() == false).TestHoodieMetricscovering both the enabled and disabled paths.Impact
Pure additive change. No on-disk state, no schema change, no behavior change for tables that disable metrics. Existing callers unaffected.
Gauge name:
<metric.prefix>.<tableName>.table.tableVersionValue: integer
HoodieTableVersion#versionCode()(e.g.6,8)Risk Level
none — additive metric only.
Documentation Update
none — surfaced through the existing reporter infrastructure; no new config.
Contributor's checklist